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I. INTRODUCTION 

For optimal control and estimation problems, the Pontryagin minimum principle 
[l] allows finding the optimal parameters of a system (controls, gains, etc.), provided the 
performance criterion is a scalar-valued function of the parameters and the state variables. 

The user of this optimization technique often experiences difficulties in choosing 
a suitable scalar-valued optimality criterion, because he may be interested in several 
cost functionals simultaneously, such as energy consumption, transfer time, and integral 
squared error, or because in vector estimation problems the estimation error is described 
more completely by the mean error vector and the error covariance matrix than by the 
trace or the determinant of the latter. 

It then is natural to consider non-scalar-valued performance criteria, e.g. vectors, 
matrices, sets, etc. The meaning of "better than" has to be defined, which mathematically 
speaking is done by introducing a partial order relation [ 4 ], [l9j, [ 2 ] in the space of the 
cost functional. For non-scalar-valued performance criteria the notion of optimality 
splits up into the more restrictive superiority and the weaker non-inferiority (Pareto 
optimality), because "better than" is not the complement of "worse than". 

In [2l], vector-valued performance criteria were introduced. In this case, a 
superior solution minimizes all of the components of the cost vector simultaneously, 
whereas for noninferior solutions, no other solution can be found, which simultaneously 
improves all of the components of the cost vector. In [6],[7], and [8], a theory of non= 
inferior solutions to optimal control problems with vector-valued performance criteria 
has been developed. In [l4], this theory has been extended to abstract partially ordered 
range spaces of the cost functional. 
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The problem of finding the more interesting superior solutions, called infimization 
problem, has not been considered in the control literature, mainly on the ground that 
one usually has contradicting cost objectives anyway. However, Ritter [l6] solved 
infimization problems of mathematical programming in abstract infinite-dimensional 
partially ordered spaces with equality and inequality type of constraints. 

The goal of this research was finding necessary conditions. in the form of an 
infimum principle for a control to be a superior solution to a dynamic infimization problem, 
where the cost criterion takes its values in a suitable finite-dimensional partially ordered 
space. Some of the results, which were obtained, were reported in [9] and [lO], 

The purpose of this paper is to publish a reasonably complete theory of superior 
solutions to optimal control problems with non-scalar- valued performance criteria, to= 
gether with some nontrivial applications. This theory is particularly useful in the analysis 
of optimal least-squares vector estimation problems (matrix-valued cost) and uncertain 
optimization problems with set membership description of uncertainty (set-valued cost). 

The most pertinent definitions and facts of the theory of partial order relations 
can be found in the paper "Necessary and Sufficient Conditions for Differentiable 
Non-Scalar-Valued Functions to Attain Extrema" [2], and they are widely used in this 
paper. Some additional facts are col lected in the Appendix. 

Sections II and 111 contain the theoretic part of this paper. In Section II, two 
optimal control problems are stated. Necessary conditions for a control to be a superior 
solution are developed for both of the problems. These new results are discussed and 
compared with the Pontryagin minimum principle [l] and with the necessary conditions 
for noninferiority (which is less restrictive than superiority is) [6], [?], [8], [m]. 
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ln Section 111, some of the sufficiency conditions for optimality for scalar- 
valued cost functionals are extended to sufficiency conditions for superiority for 
non-scalar-valued costs. 

Section IV contains some nontrivial applications. The Kaiman-Bucy filter is 
shown to be the superior solution to an optimization problem with matrix-valued cost 
(partial order of positive-semidefinite differences) [lO]. The dual problem to the filter= 
ing problem, viz. the linear-quadratic regulator problem is also discussed. Furthermore, 
an uncertain optimization problem with set membership description of the uncertainty, 
is shown to have a superior solution (partial order of set inclusion). 

The contributions of this paper to the state of the art of optimal control theory 
are discussed in the concluding Section V. 



II. NECESSARY CONDITIONS FOR SUPERIORITY 
In the statement of the optimal control problems and in the proof of the theorems, 
this section basically follows the approach taken by Halkin [ll] for optimal control prob= 
lems with scalar-valued cost functionals. Thus, the cost space is a k-dimensional partially 
ordered subspace of the (extended) n-dimensional state space. 

The statements of two dynamic infimization problems (superiority) are given in 
Section II. 1. Section 11.2 contains the results of these optimization problems in the form 
of necessary conditions for a control to be superior, which are proved in Section 11.3. 

These results are new. A preliminary report has been given by the authors in [ 1 0] . 

However, Ritter has investigated a related problem of mathematical programming in [l 6] . 
The solution to the dynamic minimization problem (noninferiority) is essentially well 
known ['6j, [7], [8], but only Neustadt [l4j considered abstract partially ordered cost 
spaces. 

In Section 11.4, the results of Section 11.2 are discussed and their relation to 
[6], [7], [8], and [l4], and to Pontryagin' s minimum principle [l] is explained. 

II. 1 Statement of the Infimization Problems 

In the statement of the optimal control problems, the cost space is a k-dimensional 
partially ordered subspace of the (extended) n-dimensional state space. Following Halkin 
[l l], the dynamics of the state are allowed to depend on all of the n state variables. The 
cost space always is a finite-dimensional integrally closed directed linear poset [2, 
Definitions 11.7, 11.9, and II.IO]. Hence, by Theorem 11.13 of [2], its positive cone is closed 


- 5 - 



- 6 - 

and has non-empty interior, and the complement of the positive cone is an open subset of 
the cost space. All finite-dimensional spaces involved are Banach spaces, but not neces= 
sariiy Euclidean spaces. This allows treatment of states, controls, and costs to be vectors, 
matrices, etc. without use of canonical transformations. All derivatives are Frechet deri= 
vatives, all measures are Lebesgue measures. 

Given the dynamical system 

x(t) = f[x(t),u(f),t] a. e. t'e [t^tj] (1) 

where x(t) e X n is the state, u(t) £ ftcX™ is the control, and te [tQ,t^]c R is the time, 
with fixed initial state x^ at the fixed initial time t^, 

X (V = x 0 ' ( 2 ) 

The final time t ^ is fixed. 

Assume, there exists £> 0, such that f[x,u,t] and 3f[x,u,t]/3x are defined, meas= 
urable with respect to u and t, uniformly equicontinuous with respect to x, and uniformly 
bounded for all (x,t,u) e N(x,t,£)x ft*, where N(x,t,t) is any closed ^.-neighbourhood 
of (x,t) e X n x [tQ,t^] and any closed and bounded subset of ft. 

The admissible control functions u are such that u(t)eft for all te [t^,^] and 
that there exists a state trajectory x : ttQ,t^] X n satisfying (2) (which is then granted 
to be unique [l l]). 

k 

The k-dimensional cost space (X,4) is an integrally closed directed linear poset 

and a subspace of the state space X . Thus, the state can be written as 

x(t) = (S(t),J(t)) (3) 

where J(t) is the component of the state, which belongs to the cost space and S(t) the 

R i 

component, which belongs to the orthogonal complement (X ,4)" of the cost space (in 
the sense of direct sum). 
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In this section, the following two infimization problems are considered. 


Infimization Problem 11.1. Find an admissible control u*, such that the final state satisfies 


the constraint 


S.(t 1 ;u*) = S 1 , 


(4) 


k J_ n 

where is a fixed element of (X ,^) c X , and such that the cost component J(t^;u*) 
of the optimal final state x(t^;u*) is infimized, i.e., 

J (t -j ; u* ) 4 J(t ] ;u) (5) 

for all admissible controls u meeting the boundary constraint, S(t^;u) = . 


Infimization Problem 11.2. Find an admissible control u*, such that the cost component of 
the final state is infimized, i.e., 

J(t 1 ;u*) 4 J(tj;u) for all admissible u. (6) 


The Problem 11.7 is a fixed-end-point problem whereas the Problem 11.2 is a 
free-end-point problem. 

11.2. Necessary Conditions for Superiority 


,n ,..k 


Convention 11.3. P(t) & ^(X' ,(X^,4)) denotes a linear map from the state space X n into 

k 

the cost subspace (X ,4). Furthermore, P(t) is decomposed into P(t) = (P^(t),Pj(t)), where 
P s (0 * /((X k ,4)- L ,(X k ,^)) and Pj(t) e /((X k ,4),(X k ,4)). 


Definition 11.4. For all t e £ t^, t ^1 , the Hamiltonian is defined by 
Hfx(t),P(t),u(t),t] = P(t)f[x(t),u(t),t] 


(7) 


where f[x(t),u(t),t] is the right-hand side of (1). Note, that H attains its value in the 

k n 

k-dimensional cost subspace (X ,4) of X . 
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Theorem II. 5. Solution to Infimization Problem 1 1.1 

In order that the admissible control u* be a superior solution to the Infimization Problem 

n k 

11.1, it is necessary, that there exist a nonzero map (P^/Pq) £ i£(X ,(X , 4 )), where Pq is 

k k 

positive (see Definition A.l), i.e., Pq 0 e ^((X ,4),(X ,4),4), such that along the 
trajectories x* = (S*,J*) : [t^,^] '♦X 0 and P* : “* 5C(X n ,(X^,4)) satisfying 


x*(t) = f[x*(t),u*(.t),t] a.e.tejt^] (8) 

x *(t Q ) = x 0 ( 9 ) 

S*(t ] ) = S 1 (10) 

p*(0 = - |^[x*(t),P*(t),u*(t),t] = - p*(t)|^-[x*(t),u*(t),t] a.e. t e [t^t^ (1 1) 

P* S (^) = P S 02) 

03 ) 


the following condition holds 

H[x*(t),P*(t),u*(t),t] 4 H[x*(t),P*(t),u(t),t] 

forallu(t)eQ and a.e. t£ [tQ,tj| , (14) 

i.e., the Hamiltonian is globally infimized with respect to u(t) along the trajectory 
(x*,P*) defined by (8) through (13). 


Remark 11.6. Unless the superior control is somewhat singular, the operator Pq can be 

k k 

chosen to be the identity operator, Pq = I £ o(((X ,4),(X , 4 ), 4 ), with no loss of generality. 
For further discussions, see Section 11.4. 




Theorem 1 1.7. Solution to Infimization Problem 11.2 

In order that the admissible control u* be a superior solution to the Infimization Problem 

k k 

11.2, it is necessary that there exist a positive map P^ > 0 e 5 \((X ,4),(X ,<),4) such that 
along the trajectories x* = (S*,J*) and P* satisfying 

x*(t) = f[x*(t),u*(t),t] a.e. t e L t Q/ 1 ^ ] (15) 

x *(f 0 ) = x 0 0 6 ) 

p*(0 = - (t), P*(t), u* (t), t] = - p*(t)^-[x*(t),u*(t),t] a.e. t e [fytj] (17) 

P^t^O (18) 

^ 1 ) = P Q (19) 

the Hamiltonian is globally infimized, i.e., 

H[x*(t),P*(t),u*(t),t] 4 H[x* (t),P*(t),u(t),t] 

for all u(t)eft and a.e. t e [tg,t^] . (20) 

Remark 11.8, Clearly, the Remark 11.6 applies to Theorem 11.7 as well as to Theorem 11.5. 

For further discussion, see Section 11.4. 

11.3. Proof of the Infimum Principle 

The proof of the infimum principle stated in Theorems 11.5 and 11.7 closely parallels 
the proof of the maximum principle for scalar-valued cost functionals by Halkin [llj. The 
results which are independent of the partial ordering of the cost subspaee of the state 
space are taken over without proof. 

The major difficulty in the proof of the infimum principle stems from the fact 
that in the real line the complement of the positive cone is a convex set, whereas in an 





integrally closed directed linear poset, the complement of the positive cone is not convex. 

The choice of notation in the following proof should allow easy referencing to [ll]. 

For an admissible control u, which fact is written ueF*, the solution of the differ= 
ential equation (l),with initial condition (2), at time t is denoted by x(t;u) and the correspond 
ing whole trajectory x for t over [t^,^] by x(u). For any admissible control u, the Frechet 
derivative D(t;u) is defined for all t 6. [tQ,t.j] by 


D(t;u) 


_ ^f(x/u(t)/i) 
3x 


(21) 


i x = x(t;u) 

By assumption, D(t;u) is bounded and measurable over [fQ/t-|]. Furthermore, the transition 
operator G(t;u) associated with D(t;u) is introduced by the operator differential equation 
G(t;u) = - G(t;u)D(t;u) a.e. te [t^j (22) 


with boundary condition at t^ 

G(t ]; u)= I, (23) 

where I : X -* X° is the identity operator. The transition operator G(t;.u) exists, is unique, 
bounded over[tQ,t^J, and invertible for all t£ [t^t^]. 

For every trajectory x(t;u*) corresponding to a u* e F*, a comoving space Y n (u*) 
with elements y is defined by 

y = G(t;u*)(x-x(t;u*)) , x £ X°. (24) 

In the comoving space Y n (u*) of a trajectory x(u*) , u* e F*, a trajectory x(u) 
corresponding to u e F* is denoted by 

y(u,u*) = {(y(t;u,u*),t) : t e [t 0 ,tj]} . (25) 

Furthermore, a "first-order approximation" 

y + ( U/U*) = {(y + (t;u,u*),t) : t e [tg,^]} 
of y(u,u*) is defined for all t£ f t^, t by 


(26) 
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p T 

y + (t;u,u*) = G(s;u*)[f(x(s;u*),u(s),s) - f(x(s;u*),u* (s),s)]ds 


(27) 


y (t;u,u*) is a first-order approximation in the sense that the sup-norm of y + (u,u*) - y(u,u*) 

q/ \ 

is bounded by a continuous, nondecreasing function 0(r), such that lim = 0, where 

r-0 r 

r ~ya({t : u(t)^ u*(t), t e [ll, p. 54] (p = Lebesgue measure). For every pair 

of admissible controls u and u*, both y(u,u*) and y (u,u*) exist, are unique, and continuous. 
Now the following “reachable sets" are introduced : 

H ={x(t 1 ;u) : u e F*} (28) 

is the set of all final states reachable from the initial state (2) by applying an appropriate 
admissible control. 

H(u*) ={y(t^;u,u*) : u € F*} for any u* £ F* (29) 

is the set H of (28) described in the coordinates of the comoving space Y°(u*) of u* e F*. 

H (u*)={y (t^;u,u*) : u£ F*} for any u* e F* (30) 

is the first-order approximation of H(u*) at y(t^;u*,u*) = 0 £ Y (u*), and is known to be 
convex for every u* a F* [l 1, p. 70 ] . Observe, that by (23) and (24), the map from H to 
H(u*) is simply a translation, which depends on u*, i.e., 

H(u*) = {x - xOyu*) : x e H} . (31) 


Proposition 11.9. If the admissible control u* is a superior solution to Inflmlzation 
Problem II. 1 or 11.2, then the point x(t^;u*) is a boundary point of the set H in X n . 


Proof : If x(t^;u*) = (S(t^;u*), J(t^;u*)) is in the interior of H, then there exists 

£ > 0, sucht that z = (S(t^;u*),J(t^;u*) - J £ ) is also in H, where (0,J fc ) & X' n has norm £, 

k 

and J £ >- 0 6 (X ,4,) is a positive element of the cost subspace. Hence, there exists a 
u z e F* satisfying the boundary constraints and resulting in the cost J(t^;u ) *< J(t^;u*). 
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Thus, x(t.j ;u* ) has to lie on the boundary of H, in order that u* be superior (or noninferior, 
for that matter [l2]). 

Remark 11.10. Clearly, by (31), every boundary point of H is also a boundary point of 
H(u*) for any u* £ F*. In particular, if x(t^;u*) is a boundary point of H, then y(t^;u*,u*) = 0 
is a boundary point of the set H(u*). 

Proposition 11.11. If the point y = 0 is a boundary point of the set H(u*), then the point 
y = 0 is also a boundary point of the set H (u*). 

Proof: See [l 1, p. 77]. 


Proposition 11.12 . If the point y = 0 is a boundary point of the set H (u*), then there exists 

n k 

a nonzero linear map P*(t^;u*) : Y (u*) -*• (X ,<), such that 


P*(t 1 ;u*)G(t;u*)[f(x(t;u*),u(t),t) - f(x(t;u*),u*(t),t)] 1? 0 
for all u(t)eft and a.e. t £ [tg,^]. 


(32) 


Proof: Since H (u*) is convex and y = 0 is a boundary point of H (u*), there 
exists a nonzero linear functional p(t^;u*) : Y n (u*) -*■ (R,<), such that p(t^;u*)y > 0 for 
all y € H (u*). Consequently, there exists a nonzero linear map P*(t^;u*) : Y n (u*) -+ 


(X ,<), such that 


P*(t^;u*)y >0 for all y e H (u*) 


(33) 


Assume, that there exists a control u satisfying the constraint u(t)eQ for all t e [t^t^], 
such that 


jj(t) = R* (t -j ; u* )G(t;u* )[f(x(t;u* ),u(t),t) - f(x(t;u* ),u* (t),t)] ^0 


(34) 


for t e E, where E is a Borel set in [t^, t of positive measure, jj.(E) > 0. In order to conclude 
from this assumption, that there exists y e H (u*) not satisfying (33), a Borel subset F £ E 
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is constructed, such that J^(t) of (34) belongs to one and the same open half-space of 

k k 

(X ,4) for all t e F, which is separated from the positive cone of (X ,4) : 

k k 

Since the complement of the positive cone of (X ,4) , {J £ (X ,4) : J <f 0}, is an 
open set, there exists £ > 0, such that the angular distance d between (t) of (34) and the 
positive cone is bigger than £ , i.e.. 


t e E ■ t £ E J 9 >0 ‘I 1 

J 2 > 0 

k k. 

where || . || denotes the norm of (X ,4). Also, since (X ,4) is finite-dimensional, there 

k ; ■ . ... ■ 

exists a closed polyhedral cone Q in (X ,4), which is circumscribed to the positive cone, 
such that the angular distance d between any element of Q and the positive cone is at 
most t , i.e., 

llJ i “ V 

sup min d(J.,J_) = sup min — rr-pr t . (36) 

J ] e Q J 2 ^ 0 ‘ 1 J ] e Q J 2 * 0 n 11 

V 0 v° 

Furthermore, sinceyu(E) > 0 and Q has finitely many faces, there exists a hyperplane II (Q), 
viz. a face of Q, such that J^(t) of (34) lies in the open half space R(I(Q)),defined by 
II (Q) and not containing Q, for all t e F, where F is a Borel subset of E of positive measure, 

/"(F) >0. 

k 

Since R(I(Q)) is an open half space of (X ,4) containing J^(t) for all t e F, 

there exists > 0, such that the distance of J-^(t) from the hyperplane II (Q) is always 

greater than Sy i.e., 

inf llJ (t)- J II> e . (37) 

t e_F 

J 2 e II (Q) 

Denoting byX(F) the support function of F, it follows from the definition of 1I(Q) and (37) 


- J, 


m 


> £ 


(35) 


inf . d(J.(t),J_) = inf min 


II J 1 (0 
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that the element 
• . AJ - 


P* (t^; u * )G(t;u* )[f(x(t;u*),u(t),t) - f(x(t;u*),u* (t),t)]dt 


(38) 


t 6 F 

k 

lies in the complement of the positive cone of (X ,4) together with a ball of radius £.^(F) 
and center AJ, hence that the admissible control [ll, p. 45] 

v(t) = u*(t) +X(t)(u(t) - u*(t)) (39) 

-j- ■ 4* 

generates y (t^;v,u*) e H (u*), such that 

P*(t 1 ;u*)y + (t 1 ;v,u*) = AJ ^ 0 , (40) 

contradicting (33). 

Thi s concludes the proof of Proposition 11.12. 


Since X and Y (u*) are isomorphic, viz. by the translation relationship (31), 
P* (t^;u* ) can be split up into P*(t^;u*) = (P^(t^;u*),P* Oyu*)), where P^(t^;u*) e 

sC«X k >),(X k ,<j,-«) and P|(t 1 iu*) a'it«X k ,<Jr L ,(X k ^)) . ■ 


Proposition 11.13. In Proposition 11.12, the linear map P*(t^;u*) is such that P*(t^;u*) 
is a positive map, i.e., 

P* J (t I ;u*)i.0«rf((X k , < ),(X k ,<),^) . (41) 

Proof : In the case of the free-end-point Infimization Problem 11.2, the set H (u*) 
must not intersect the open subset 

V={y=(S,J)£YV).: J^O, S ^(X^) 1 } (42) 

of Y (u*), because otherwise a v e F* could be found, which would be noninferior to u*, 

“f 

since H (u*) is the first order convex approximation of H(u*) at y(t^;u*,u*) = 0 Y (u*). 
Hence, 

H + (u*)c{y = (S,J)eY n (u*): J ►<>, S£(X k ,4) i l (43) 

n k 

and therefore every linear map Pe<^(Y (u*),(X ,4)) of the form P= (P^,Pj).with 
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P s = 0 e J e((X k ^r / (X k ,<)) (44) 

and Pj ^ 0 £ e£((X k ,4),(X k ,4),<) satisfies 

Py > 0 for all y e H (u*) . (45) 

In the case of the fixed-end-point Infimization Problem 11.1, the interior of the 
set H (u*) must not intersect the subset 

V = {y = (S,J) € Y n (u*) : J * 0, S = 0} (46) 

of Y (u*), because otherwise a v e F* could be found, which would be noninferior to u* 
since H (u*) is the first order convex approximation of H(u*) at y(t^;u*,u*) = 0 eY n (u*). 
Hence, H (u*) is contained in a set A of the form of the vector sum 

A = B © C , (47) 

where B = {y = (S,J) £ Y n (u*) : S = 0, J >0}and C is an n-k dimensional subspace of 
Y (u*) which is separated from B. 

If B and C are disjoint, except at the origin, rather than merely separated, the 
Infimization Problem II. 1 is called regular or nonsingular. In this case, every linear map 
P £ ^(Y (u*),(X K ,=0) of the form P = (P^/Pj) with J((P) = C and Pj = I satisfies (45) for all 
y e H + (u* ) 

If B nC contains more than the origin, i.e., if B and C are separated but not dis= 

joint, the Infimization Problem II. 1 is called singular. In this case, there exists a linear 

k k 

map Pq y 0 e s£((X ,4),(X ,4),4) (but Pq / I), such that every linear map P e ^(Y (u*), 
(X ,4)) of the form P = (P<.,Pj) with Jf(P) 3 C and P^ = Pq satisfies (45) for all ye H (u*) . 
This concludes. the proof of Proposition 11.13. 
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Remark 11.14. if the linear map P*(t;u*)e ^\(X n / (X^,<)) is defined by 

P*(t;u*) = P*(t ;u*)G(t;u*) (43) 

then by (22) and (23), P*(t;u*) satisfies the operator differential equation 

P*(t;u*) = - P*(t;u*)D(t;u*) a.e. t £ [t^] j (49) 

with boundary condition 

P*(t;u*) = P*(t^;u*) for t = t^ . (50) 

Furthermore, with the definition of the Hamiltonian (7), the equation (32) of Proposition 
11.12 can be written as 

H[x*(t),P*(t;u*),u*(t),t] < H[x*(t),P*(t;u*),u(t),t] 

for all u(t)eii and a.e. t e [tQ,t^] . (51) 

Remark 11.15. Combining Proposition 11.9, Remark 11.10, Propositions 11.11, 11.12, and 
11.13, equation (44), Remark 11.14, and equation (21) completes the proofs of the 
Theorems 11.5 and 11.7. 

11.4. Discussion 

The infimum principle in the two Theorems 11.5 and 11.7 is stated fora globally 
superior control u* as required in the Infimization Problems II. 1 and 11.2, respectively. 

.In the proof of the infimum principle in Section 11.3, the analysis is global with respect 
to the control u(t)eft as expressed in (32), (14), and (20). However, the analysis is only 
local in the state space X and in the space of comoving coordinates Y n (u*), since the 
first order convex approximation H (u*) of the reachable set H(u*) is investigated 
((26) through (30)). 

Therefore, the infimum principle also applies to a control u*, which is locally 
superior in the following sense: 
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Definition 1 1 . 1 6. An admissible control u* is locally superior, if there exists £>0, such 
that for every admissible control u satisfying the boundary constraint at t^ (i.e., (4) in 
Problem II. 1) and generating x(t^;u) in the ^-neighbourhood of x(t^;u*) (i.e., 

i[ x(t,;u) - x(t^;u*)ll < £■), the cost component J(t^;u) of x(t^;u) is related to the locally 
superior cost J(t^;u*) by J(t^;u*) 4 J(t.;u). 

Clearly, a globally superior control is also locally superior but not vice versa. 
Therefore, when applying the infimum principle to solving infimization problems, the 
giobality of the superior solution has to be verified separately, in addition to investigating 
the existence of a superior solution. 

In Theorems 1 1.5 and 11.7, the costate P(t) was chosen to be a linear map from 
into the cost-subspace (X^,4) of X P , i.e., P(t) e s£(X P ,(X k ,4)). Now, for j , 1 £ j < k, 
let (X*,4) denote a [-dimensional integrally closed directed linear poset, in particular 
for j = 1, (x\<) = (R,4). If u*, x*, and P* satisfy the infimum principle, then for every 
positive map P 1 6 ^((X k ,4),(X*,4),<) , P^VO, 

P^x*^),?*^),^^)^] = P 1 P*(t)f[x*(t),u*(t),t] 

4 P 1 H[x*(t),P*(t),u(t),t] = P^P*(t)f[x*(t),u(t),t] 

for all u(t)e£2. and a.e. t« [tQ,^] . (52) 

By the linearity of the costate differential equation (11) or (17), respectively, this implies 

that the costate map P(t) could have been chosen in s((X ,(X*,4)) for any 1 £ j < k rather 
n k 

than in s((X ,(X ,4)). However, the infimum principle for j < k would be weaker than that 
of Theorems 11.5 and 11.7, because the truth of (52) for a particular positive map P^does 
not, in general, imply the truth of (52) for all positive maps P^ (see Lemma A. 2). As a 
matter of fact, the conditions obtained for j = 1 merely constitute necessary conditions 
for noninferiority [6], [7], [8], [l4]. 
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In the proof of Proposition 11.13 and in the Remark 11.6, an infimization problem 
has been called regular or nonsingular, if Pq in (13) or (19), respective!/, can be taken to 
be the identity operator, otherwise the infimization problem has been called singular. In 
optimal control theory for scalar-valued cost functionals, this type of singularity of an 
optimal control problem is well known [l],’[l3]. In this case, the nonnegative constant 
pQ happens to be zero, it can be expected that singularity of superior controls does or 
does not occur under conditions quite analogous to the conditions, under which singularity 
of optima! controls for scalar-valued cost functionals does or does not occur, respectively. 

In the statements of the Infimization Problems II. 1 and 11.2 in Section II. 1, the 

k n 

space (X ,4) has been chosen to be a subspace of the state space X , and the dynamics (1) 

of the system have been allowed to depend on all components of the state x(t) = (S(t),J(t)) 

(3). This is the most general case. 

In the important special case, where f[x,u,t] in (1) actually does not depend on the 
cost component J(t) of the state x(t), the component Pj(t) of the costate P(t) = (P^(t),Pj(t)) 
(Convention 1 1.3) is constant, hence, Pj(t) = Pq in Theorems 11.5 and 11.7. It then is more 
convenient to use the following convention : 


Convention 11.17. The cost space is a k-dimensional integrally closed directed linear poset 

k n k 1 

(X ,^), the state space is an n-dimensional linear space X (previously denoted by (X ,<) ), 

k n 

of which (X ,■<) is not a subspace. The state is x(t) £ X (previously S(t)). The costate 

n k 

P(t) is a linear map from the state space into the cost space, i.e., P(t) £ <£(X ,(X ,X)). 

k k 

In addition, there now is a constant linear map P^ e s(((X ,4),(X ,4),$) (previously 
in Theorems 11.5 and 11.7). 


In this special case and using Convention 11.17, the Infimization Problem II. 1 is 
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restated as follows 


Infimization Problem II. 18. Given the system 
x(t) = f-j^x(t) / u(t) / tj a.e. ffiftgrtj] 
*(tq) = x 0 
x(t ] ) = x 1 , 


(53) 

(54) 

(55) 


find an admissible control u* (in particular, u*(t)e.ft for all t £ [t^t^]), such that the 
cost J(u) defined by 


^1 


J(u) =, 


, f 9 [x(t),u(t),f]df 

0 


(56) 


is globally infimized, i.e., 

J(u*RJ(u) 

for all admissible controls u meeting the boundary constraint x(t^) = x^. 


(57) 


Of course, the statement of the Infimization Problem 11.2 can be adapted in a 
simi lar way. 

With the new Hamiltonian 

H[x(t), P Q,P(t),u(t),t] = P 0 f 2 [x(t),u(t),t] + P(t)f 1 [x(t),u(t),t] (58) 

instead of (7), Theorem 1 1 .5 applied to the Infimization Problem 11.18 becomes 


Corrolary 11.19. In order that the admissible control u* be a superior solution to the 
Infimization Problem 11.18, it is necessary that there exist a nonzero map (P*(t^),P^) 
e i(X n x (X^,<) / (X^,4)) with P* positive, such that along the trajectories x*, P* satisfying 
X*(t) =.f 1 [x*(t),u*(t),t] . a.e. t fc [t^t^ ■ • (59) 

x* (t Q ) = x Q (60) 

(61) 


x* (»,)•= x 1 
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(t) = " 17 l x * (t)/p o ,p * (t)/U * 

3f 9 3f l 

= - (t),u* (t),t] - P* (t) - 3 ^ [x* (t), u* (t), t] a.e. t e [t^l (62) 

the following condition holds 

H[x*(0,Pq,P*( 0/U* (t),t] < H[x*(t),P*,P*(t),u(t),t] 

for all u(t) eft and a.e. t £ [t^t^j , ■ (63) 

i.e., the Hamiltonian is globally infimized with respect to u(t) along the trajectory (x*,P*) 
defined by (59) through (63). 


Of course, the Corollary 11.19 specializes to exactly the type of theorems given 


in [l] for the case k = 1 (scalar-valued cost), in the nonsingular case, again, P* = I 
4 ^((X k ,0,(X k ,^),<). 


The reader should have no trouble in extending the necessary conditions for a 
control to be optimal with respect to a scalar-valued cost [l]in problems with other 
boundary conditions or other types of cost functionals than those reported here, to the 
corresponding set of necessary conditions for a control to be superior with respect to a 
non-scalar-valued cost (transversality conditions, final state penalty terms, free final 
time, etc.). 



III. SUFFICIENCY RESULTS 

. The purpose of this section is to show how some of the known results in the theory 
of optimal control for scalar-valued cost functionals concerning sufficiency conditions for 
a control to be optimal [l 3] can be extended to the theory of superior controls for non- 
scalar-vaiued performance criteria. 

In Section III.l, a Hamilton-Jacobi-Bel Iman type of theory is developed, which 
constitutes a sufficiency condition for a control to be superior relative to a region Z in 
the product of the state space X n and the time axis R. This theory is quite analogous to 
the Hami Iton- Jacobi-Bel Iman theory in optimal control with scalar-valued cost. 

In Section III.2, some sufficiency results for global optimality are generalized to 
sufficiency results for global superiority. 

On the other side, all of the non-trivial existence results of optimal control for 
scalar-valued cost (e.g. [l3, pp. 259 ff. and 286 ff.] and [5]) generalize to existence 
results for the less restrictive noninferior controls (see [15]) rather than for the more 
restrictive superior control, in the case of non-scalar-valued performance criteria. 

III. 1. Hami Iton- Jacobi-Bel Iman Theory 

Consider the dynamic system 

x(t) = f[x(t),u(t),r] (I) 

where the state x(t) £ X° t the control u(t) is restricted to a closed subset Q. cX™ , and 
f : X°x Q x R — X n is continuously differentiable on its domain. A control u is admissi= 
ble, if it is piecewise continuous and satisfies u(t) e £2 for all t of interest. 

It is assumed, that for every admissible control u and any initial state 
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x (*q) = Xq / (2) 

there exists an absolutely continuous solution x satisfying (1) a.e. and (2), (which then 
is automatical ly unique by the local Lipschitz continuity of f with respect to x(t)). 

The target set S is a subset of X n x {t e R j t > t^. The non-scalar-valued cost 
to be infimized is of the form 


J(u) = K[x(t ),t] +. J l[x(t),u(t) / t]dt 
11 0 


( 3 ) 


K 

an.' attains its value in the integrally closed directed k-dimensional linear poset (X ,4), 
which may or may not be a subspace of the state space X n . Here, L and K are continuously 
differentiable on their respective domains, and the final time t^ > t^ is the first time, the 
target set S is met by the trajectory x generated by the admissible control u. 


Convention I II. 1. Z denotes a connected subset of X° * R, which intersects the target 

— n k 

set S. Furthermore, II always denotes a connected subset of ^(X ,(X ,4)) x R. 

Definition III. 2. The Hamiltonian H : x pC(X n ,(X^,^)) x SI * R '♦(X ,4) defined by 

H(x,P,u,t) = L(x,u,t) + Pf(x,u,t) (4) 

is called normal relative to Z and II, if for each (x,P,t) with (x,t) £ Z and (P,t) e II , the 
Hamiltonian has a unique absolute infimum with respect to all ue£l, viz. at the point 

u = u^(x,P,t) e£l, (5) 

which is called the H-infimal control relative to Z and II. 


Now, the following theorem can be stated, which is the analogue to the 
Hami Iton- Jacobi-Bel Iman theory of optimal control for scalar-valued cost functionals 
[l, p. 351], [3, p. 315]. 
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Theorem !!l.3. Suppose that the Hamiltonian (4) is normal relative to regions Z and II and 


let u (x,P,t) denote the H-infimai control relative to Z and II.. Suppose that u* is an 
admissible control, which transfers (xq/^q) e Z to S, such that the corresponding trajectory 


x* stays entirely in Z, i.e., 

(x*(t),t) £ Z for all t e [t^] . (6) 

Suppose that there exists a continuously differentiable function J(x,t) on Z satisfying 
the partial differential equation 

|jr(x,t) + H[x,~(x / t) / u°[x / ^(x f t),t] / t] = 0 (7) 

with boundary condition 

J(x,t) = K(x,t) for (x,t) e SnZ , (8) 

such that ^(x/t) stays entirely in II , i.e., 

(-j— (x,t),t) £ II for all t £ • (7) 

If 

u*(t) = u°[x*(t),|i(x*(t),t),t] for all t 6 [t^t^] , (10) 


then u"' : LtQ, t ] — • A Is a superior control relative to the set of admissible controls 
u generating trajectories x and lying entirely in Z and II, respectively, and the 
infimal cost is given by 

J(u*) = J(x 0 ,t Q ) 4 J(u) for all u e . (11) 


Remarklli.4. Clearly, if the Theoremlll.3 applies to the case of the initial point (Xq^q) 

£ Z, then, it also applies to any initial point (x*(t),t) with t £ [tQ,t^land the corresponding 


"cost to go" J(x*(t),t) . 
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Corollary 111.5. If Z = x" * [t ,t.], and S Z, and II = £(X n ,(X^,^)) * [t ,tj, then the 
Theorem 111.3 provides a sufficiency condition for global superiority for all initial states 

( x o' f c)) 6 z * 

Corollary III. 6. If in cddition to the assumptions of Theorem 111.3, the function J(x,t) is 
twice continuously differentiable on Z, Z is open in X x R, and the H-infimal control 
u ^[x, -g— (x,t),t] is continuously differentiable with respect to x and continuous in t, then 
the function P : [t^,t ] sC(X n / (X^,4)) defined by P(t) = ^-(x*(t),t) is a costate function 

in the infimization problem (1), (2), (3). This follows directly from the corresponding 
proof by Kalman in [3, p. 320] at least in the case where 0-= X™. 


Proof of Theorem 1 1 1.3. With x = x*(t), (7) becomes 


SJ (x*(t),t) + H[x*(t) / ^-(x*(t) / t) / u°[x*(t),^-(x*(t) / t) / t],t] 


3t " L ‘ ' "Sx'” w/ 5x 

_ 4"( x *( t ) ,t ) + L{x*(0/.u 0 ix*(t) / |j(x*(t),t) / it],t] 


j-(x* (t),t)f [x* (t),u U [ x* (t), |^(x* (t),t),t],t] 


3x 


ax' 

3J 

3x 


— (x*(0,0 + L x* (t),u°Lx*(>), = 0 . 


( 12 ) 


Integrating (12) from t^ to t^ and using (8) yields 




J(x*(t 1 ),t 1 ) - J(x o rt Q ) + 


l[x* (t),u°[ x * (0, (0/0fri/ 


dt 




= K(x*(t^),t^) - -l(x 0 ,t 0 ) + 


X* (t),u°[x* (t), (x* (t),t), t],t 


dt 


= J(u*) - -l(x 0 ,t 0 ) - 0 


(13) 


For any admissible control u : transferring (xq^q) to (x e S, such that 
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the trajectory x^ : [t^t^] — X n lies entirely in Z and '|~( x u (*)/*) : 3C(X n / (X* < / <)) 

lies entirely in II, equation (7) reads at every t e anc * w '^ x = x (0/ 


|^(x u (t),t) + H [ X y (t)/ ( x u (t), t), u°[x^(t), (x^ (t), t ), t], t] = 0 


9J 


Or 


And by the normality of H, (14) yields 


-ljr(x (t),t) + H[x (t),-H(x (t),t),u(t),t] 
ot u u u dx u 


( 14 ) 


= 4~( x (0/0 + L[x (t),u(t),t] + ^(x (t),t)f[x (t),u(t),t] 
ot U U dx u u 

V 0 , whenever u(t) ^ u^x^O/'l^x^t^t^t] 

= 0 , whenever u(t) = u*\x (t),-^-(x (t),t),t] 

u ox u 

Integrating (15) from t^ to t^ and using (8) then results in 
J(u) - J( x 0 /f Q ) > 0 . 

Subtracting (13) from (16) results In 

J(u) - J(u*) <*• 0 for all u £ U^-j^v 

Combining (13) and (17) completes the proof of (11) and of the Theorem 111.3. 


05 ) 


06 ) 


( 17 ) 


1 1 1.2. More Sufficiency Results 

In this section a sufficiency result given by Lee and Markus [l 3, p. 34 1 ] is 
generalized to the case of non-scalar-valued performance criteria. 

Consider the. dynamic system 

*(0 = A(t)x(t) + . h(u(t),t) , a.e. , (18) 

where the state x(t) e X°, the control u(t)e Q c X m is measurable, A(t) e ^C(X n ,X n ) is 
continuous in t over [t^/t^, and h : ft x [t ,tj] ”*■ X* 1 is continuous on its domain. The 
initial state at the fixed initial time t^ is 
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x(t o ) = x o ; 


09 ) 


The target set S c )C at the fixed final time t^ is closed and convex (and possibly S = X n ), 


x(t^) € S . 


(20) 


The non-scalar-valued cost to be infimized is of the form 

,st. 

J(u) = 


J 


1 


(f°(x(t) / t) + h°(u(t),t))dt 


( 21 ) 


0 


k 0 

and attains its value in rhe integrally closed directed linear poset (X ,4). In (21), f , 

3f /3x, and h^ are assumed to be continuous on their domains. Furthermore, f^ is 
assumed to be convex in x (Definition 11.19 of [2]) for all te [tg,^], i.e., 
f 0 (s x i + (1 - s)x 2 ,t) 4 sfVi't) + (l-s)f°(x 2 ,t) 

for all s £ [0,1], all x^ and x^ in X n , and all t £ [tg,^] . (22) 


Theorem 11,1.7. If the optimal control problem is nonsingular and if there exists a measur= 
able control u*: [tQ,t^]-*Sl satisfying the necessary conditions of the infimum principle, 
then u* is globally superior (although not necessarily unique), provided that one of the 
following conditions holds : 

a) S = X n or x*(tj) e int(S) 

b) x*(t^) £ 9S and P*(t^)(x - x*(t^)) < 0 
for all x £ S, which are reachable at t^ 

c) S={x 1 } . (23) 

k k 

Proof. Since the optimal control problem is nonsingular, Pq e ^((X ,4),(X ,4),4) 

can be taken to be the identity map, Pq = I, by Remark 11.6. Hence, the Hamiltonian is 

H(x,P,u,t) = f^(x,t) + h^(u,t) + PA(t)x + Ph(u,t) . (24) 

0 k 

Now, define the quantity x (t) £. (X ,4) for all t £ [t^t^l by 
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x°(t) = f°(x(0,t) + h 0 (u(t),0 a.e. t * (25) 

x°(t 0 ) = 0, (26) 


where x : [t^,^] X n is the trajectory generated by the admissible control u : [t^t^-*^, 
and consider the quantity 

x°(t) + P*(t)x(t) * (X k ,=0 (27) 

where P*(.) is the optimal costate, which satisfies 
' • - 3 f° 

■ P*(0 = - . ( x * (t-), 1 ) - P*(t) A (t) a.e. t e [t^t^ (28) 

^(t^ P* , (29) 

according to the infimum principle. 

Differentiating (27) and using (18) and (28) 

-nr(x°(t) + P*(0x(0) = x°(t) + P* (t)x(t) + P*(t)x(t) 

.0 

= f°(x(t),t) + h°(u(t),t) - -(x*(t),t)x(t) + P*(t)h(u(t),t) . (30) 

Integrating (30) with respect to t over [1^,1 ^ yields 


x (tp + P^t^x^) - P*(^q)x 0 


1 


{ f°(x(t),t) + h°(u(t),t) - -~(x*(t),t)x(t) + P*(t)h(u(t),t)}dt . (31) 


'0 

Evaluating (31) for u* and its corresponding state x* and subtracting this equation from (31), 
x°(t ] ) - x°'(t, ) + P*(t ] ')(x(t ] ) - X* (t, )) 

, 0 

(' (x(t),0- f°(x*(t),tj- |E(x*(t),t)(x(t)-x*(t)) 


c/x 


0 


+ h°(u(t),t) + P*(t)h(u(t),t) - h u (u*(f),t) - P*(t)h(u*(t),t)}dt . (32) 


0 


.0 


The convexity and the differentiability of f in x together with Lemma 11.20 of [2] imply 
f°(x(t),t) - f°(x*(t),t) - -^-(x*(t),t)(x(t) - x*(t)) V0 for all t e [t^] . (33) 



- 28 - 


3 y hypothesis, u* globally infimizes the Hamiltonian H(x*(t),P*(t),u,t) for a^e. t e [t^,^], 
which implies 

h°(u(t),t) + P*(t)h(u(t),t) - h°(u*(t),t) - P*(t)h(u*(t),t) > 0 
for a.e. t £ [t^,^] . (34) 

Combining (32), (33), and (34) yields 

X°(^) - x®*(tj) + P A (t 1 )(x(t 1 ) - x*(t^) ^0 . (35) 

In case (a), where S — X or x*(t^) £ int S , necessarily P*(t^) = 0 in (29) and (35). Hence, 
J(u) - J(u*) = x^(t^) - x^* (t ^ ) i*-0 for all admissible u . (36) 

In case (b), where x*0y)e 3S, the hypothesis P*(t^)(x- x*(t^))4 0 for all x £ S, which 
are reachable at t^ combines with (35) fo (36). In case (c), where S ={x^}, only the 
controls u meeting the boundary condition x(t^ ) = xj = x*(t^) are admissible. Thus, (35) 
reduces to (36) again. 


If in the optimal control problem (18), . . . , (21), the target set S = X and the non- 


scalar-valued cost to be infimized is of the form 

%t. 


J(u) = K(x(tp) + 


1 


(f°(x(t),t) + h°(u(t),t))dt 


(37) 


0 


rather than (21), where K : X n -*■ (X^,4) is convex and differentiable on X n and f^ and 
h^ have the same properties as they have in (21), then the following sufficiency result 
is obtained : 


Theorem 111.8. If the optimal control problem is nonsingular and if there exists a 
measurable control u* : [t^,^] -*> SI satisfying the necessary conditions of the infimum 
principle, then u* is globally superior (although not necessarily unique). 

Proof. As in the proof of Theorem III. 7, with P* = I, and with the definitions (25) 
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and (26), the inequality (35) is obtained. Since necessarily 

p* ( >,)=!f(x* ( t,)> - < 3s > 

ecuction (35) reads 

x 0 ( , ,)-x 0 *(t 1 )+-^(x*(t,))(x(t ] )-x*(t ) ))VO . (39) 

Adding and subtracting identical terms in (39), viz. K(x(t^)) and K(x*(t^)), respectively, 

(39) becomes 

x°(t,) + K(x(tj)) - X°*(t,) - Kfx'ttj)) - K(x(r 1 )) + K(x*(t^) 

+ ^(x* ( t ] ) Kx ( t ) )-x*(t, )) ^ 0 . (40) 

By the convexity of K, the sum of the last three terms in (40) is in the negative cone 
of (X^,4). Thus, 

J(u) - J(u*) » x°(t,) •+ Kfxjtj)) - x°*(t,) - K(x*(t,» 

for all admissible a . 


( 41 ) 



IV. APPLICATIONS 


In this section, the infimum principle is applied to nontrivial infimization problems. 

In Section IV. 1, the Kalman-Bucy filter is rederived as the superior solution of a dynamic 

optimization problem with a matrix-valued cost criterion, viz. the error covariance matrix 

at some final time. In Section IV.2, an infimization problem is discussed, which is dual to 

the Kalman-Bucy filtering problem. Furthermore, the so-called separation theorem for a 

stochastic linear control problem with a quadratic cost functional is discussed. In Section 

IV. 3, a superior solution to an uncertain optimal control problem with set-valued cost is 

found, where the uncertainty is described by set membership. 

In this section, M denotes the linear space of n by m matrices with real entries, 
nrn 

M the abridqed notation of M , M 1 the linear space of n by n symmetric matrices with 
n nn n 

real entries, and (M^,=S) the linear space partially ordered by positive-semidefinite 
differences (Example II. 3 of [2]). 

Definition IV. 1. The linear operator U : M ”* M‘ is defined for al I A £ M by 
— ; r n n n 

UA - A + A' . (1) 

Definition IV. 2. The linear operator T : M M is defined for all A e M by 
nm mn nm 

TA = A' ' (2) 

Obviously, by Definitions IV. 1 and IV.2, U and T satisfy the operator equality 
UT = TU = U , (3) 

provided dimensions match. 
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IV. 1. Rederivation of the Kalman-Bucy Fi Iter 

A slightly simplified version of the problem considered below has been presented 
by the authors in [lO]. 


Statement of the Infimization Problem 

Consider the n-th order linear time-varying dynamic system 

x(t) = A(t)x(t) + B(t)u(t) • (4) 

x(t 0 ) = x 0 • (5) 

The m-vector input u(.) is a white stochastic process with 

E{u(t)} = 0 for all t (6) 

E{u(t)u'(s)} = Q(t)£(t-s) , (7) 

where Q(t) > 0 £ (M 1 ,4) for all t. The initial state x n is a random vector with 
m U 

■ -o (8) 

E;(x 0 - *c ''*0 - ^ 0 6 • ( 9 ) 

The p-vector observation 

y(t) = C(t)x(t) + v(t) (10) 

is corrupted by a white stochastic process v(.) with 

E{v(t)} =• 0 for all t (11) 

E{v(t)v'(s)}=R(t)S(t-s) , (12) 

where R(f)£ 0 e (M 1 ,4) is a positive-definite matrix for all t. The noises u and v are 
P 

assumed to be correlated according to 
E{u(t)v‘(s)} = S(t)S(t-s) , 

where S(t) 6 M is such that the auqmented matrix 
mp 


(13) 


Q(t) s(t) 

L S‘(t) R(t) j 


y 0 € (M, ,y .,4) 

' . (m+p)(m+p) 


(14) 




is positive— torn idof : n i te for a!i f. Furthermore, the random initial state x_ is assumed to 

0 

be independent of both u(.) end v(.) . 

If is desired to find cn n-fh order linear unbiased estimator with infimal conditional 
error covariance matrix at some given final time t^ > t^. In other words, the error e(t) 
between the true state x(t) and the state estimate w(t) by the n-th order I inecr time- 
varying dynamic filter 

w(t) = F* (t)w(t) + G Jt (t)y(t) (15) 

06 ) 

has to have conditional mean zero, i.e.. 


w(t 0 ) = W* 


Ele(t) | y(s), t Q « s s >i= E{x(t) - w(t) I y(s), s s « t} = 0 for all t , (17) 

and F* : [t-,t,] -* M , G* :[t 0 >tj -* M , and w* e R n have to be found such that 
UlnUf np U 


Z*(t ) = E{e(t )e'(t ) ] y(s), t.s s s t] 


-| / “ l-\- J /~ V ■] / | / \»// ‘Q 

* £(fj) = E{e(t 1 )e'(t 1 ) | y(s), t Q < s c t} 
for all other choices of F, G, and w_. 


F*,G*,w* 


F, G,w. 


(18) 


For a discussion of the matrix-valued performance criterion, see [2, Remark IV. 2]. 


Analysis of the Necessary Conditions of the Infimization Problem 

For arbitrary choice of F, G, and Wq, the estimation error e(t) = x(t) - w(t) satisfies the 
differential equation 

e(t) = (A(t) - F(t) - G(t)C(t))x(t) + F(t)e(t) + B(t)u(t) - G(t)v(t) (19) 


e(t o ) = x o ' w o • 


(20) 


Requiring unbiased estimates for all t 6 y‘ e ^ s 


w o x o 


( 21 ) 
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and 

F*(0 = A(t)-G*(0C(0 (22) 

since u and v ere zero-mean and E{x(t)}/ 0 in general. Replacing F(t) by A(t) - G(t)C(t) 
in (19) and with (21), the error differential equation (19), (20) becomes 

e(t) = (A(t) - G(t)C(t))e(t) + B(t)u(t) - G(t)v(t) (23) 

e(t Q ) = x(t Q ) - x Q . (24) 

In order to obtain a deterministic optimal control problem, the error covariance matrix 
I(t) is introduced 

I(t) = E{e(t)e'(t) | y(s), t Q £ s < t} , (25) 

which satisfies the matrix differential equation 

t(t) - U(A(t)- G(t)C(t))Z(t) + B(t)Q(t)B'(t) + G(t)R(t)G'(t) - UB(t)S(t)G'(t) (26) 

r<f o ) = I o ' (27) 

In the remaining deterministic infimization problem, Z(t) is the (extended) state and G(t) 
is the control. The cost to be infimized is , 


J(G) = I(t 1 ) . (28) 

In this problem, the cost space is the entire state space (of dimension n(n+l)/2). The 
costate P(t) belongs to /((M^,<),(M^,4),^) and the Hamiltonian is 

H = P(t)Z(t) . (29) 


By Theorem 11.7, if G* : [t^t^] 


M is superior, then the following relations hold: 
np 


Z*(t) = U(A(t) - G*(t)C(t))Z*(t) + B(t)Q(t)B'(t) 
+ G*(t)R(t)G*'(t) - UB(t)S(t)G*'(t) 

W aZ o- 


(30) 

(31) 


P*(t) = - P*(t)U(A(t) - G*(t)C(t)) (32) 

P*^) = P Q * 0 fe ^((M^,<),(M^,4),4) (33) 
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H(xn0,P(4G*(0,t) ^ H(Z*(0,P*(t),G(t),t) 

for all t & [t A/ t,] and all G(t) 6 M 

U i np 


(34) 


Since no singularity condition arises, Pq in (33) could be taken to be the identity map 
by Remark 11.6. 

Observe, that the homogeneous differential equation in M 1 

t< t) = U(A(t) - G(t)C(t))Z(t) (35) 

i(t 0 ) = Z 0 (36) 

has a positive-semidefinite solution Z(t) for all t, whenever Zq is positive-semidefinite. 


"^(0 (^o^O 


(37) 


where the transition operator is the solution of the operator differential equation 

VJ7 


£* q (V 0 ) = U(A(t) - G(t)C(t))$ G (tt 0 ) 

^VO' = 1 ' 


(38) 

(39) 


Hence, the transition operator (t,t_) 6 < /((M’,4),(M 1 ,4),4) is positive (Definition A.l) 

O U n n 


for all t e f t^, t ^ and all possible choices of G : ^ 


M 


np 


Now, since the solution of the costate differential equation (32), (33) is 

?'W = P 0 9 G ,(t,,;) , WO) 

it follows, that the costate P*(t) is positive for all t and all Pq > 0. Therefore, the 
infimization of the Hamiltonian (29), (34) is achieved by infimizing Z(t), hence, 

U(A(t)- G*(t)C(t))I*(t) + G*(t)R(t)G*'(t) - UB(t)S(t)G*'(t) + B(t)Q(t)B'(t) 

4 U(A(t)-G(t)C(t))1* (t) + G(t)R(t)G'(t) - UB(t)S(t)G'(t) + B(t)Q(t)B*(t) 


for all t e [t A ,t ] and all G(t) £ M 

U I np 


(41) 


Since Z(t) is quadratic in G(t) with positive-hemidefinite ([2], Definition 11.18) second 
Frechet derivative (by the positive-definiteness of R(t)), it is necessary and sufficient. 


! 
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that the first Frechet derivative of i*(t) with respect to G(t) vanish at G*(t) in order 
that Z *(t) attain an infimum (by Theorems 111.1, and 111.2, and Remark 111.3 of [2*])-. Thus, 
with (3) 

= U(G*(t)R(t) - I*(t)C(t) - B(t)S(t))T = 0 , (42) 

G = G* (t) 

which implies 

G*(t) = (Z*(t)C , (t) + B(t)S(t))R" 1 (t) . (43) 

Solution to the Infimization Problem 

The superior n-th order linear unbiased filter for the plant (4), ... , (14) is 

w(t) = A(t)w(t) + (Z(t)C'(t) + B(t)S(t))R _1 (t)(y(t)- C(t)w(t)) (44) 

w(t Q ) = x Q (45) 

and the error covariance matrix X(t) e (M^,4) is precomputable from the matrix Riccati 
differential equation 

i(t) = (A(t) - B(t)S(t)R _1 (t)C(t))I(t) + I(t)(A(t) - mS(t)R\t)C(t)y 

+ B(t)Q(t)B'(t) - I(t)C'(t)R”^(t)C(t)Z(t) - B(t)S(t)R~\o$ , (O.B , (f) (46) 

= ' ( 47 ) 

This result and the fact that the error covariance matrix Z(t^) is infimized are 
well known[l8], at least for S(t) s 0. However, the derivation is new. The special case 
p = m with v(t) «* u(t) may be of some interest, if only for demonstration purposes, when 
only one noise generator is at hand. In this case, R(t) = Q(t) = S(t) = S'(t) for all t. 

Sufficiency Analysis 

In the sequel it is proved, that the Kalman-Bucy filter indeed is a globally superior solution 
to the infimization problem. If in (13), the matrix S(t) is set identically equal zero for the 
sake of simpler arithmetics, then, the deterministic Kalman-Bucy filtering problem consists 


3i*(0 

aG 
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of the dyncmic system 

Z(f) - (A(r) - G(t)C(t))X(t) + 2(t)(A(t) - G(t)C(tj)‘ 

+ B(t)Q(t)B‘(t) + G(t)R(t)G'(t) (48) 

^o ) = x o (49) 

and the matrix-valued cost 

I ' . 

J(G) = X(t 1 ) . (50) 

In the context of the Hami Iton-Jacobi-Bel Iman theory of Section III.l, this problem is 
interpreted as follows : In (3) of Section 111 

K(Z(t 1 ),f 1 )=-Z(t 1 ) (51) 

L(Z(t),G(t),t) s 0 . (52) 

The target set is S = (M\<) * {t^, the regions Z = (M‘^,4) x {t e R | t £ t,A and 

II = (positive cone of ,4),4) x {t e R J t £ t^. The Hamiltonian (4) of Section 

III is 

H(I,P,G,t) = P((A-GC)I + I(A-GC)‘ + BQB 1 + GRG') (53) 


where the notation of the time dependence Is suppressed. Since the costate P(t) with 

9 K 

boundary condition P(tp = -^=-(Z(t^),tj) = I necessarily is a positive map for all t £ t^ 
the H-infimal control is 

G°(I / P / t) = ZC , (t)R _1 (t) . (54 


With (54), (48) and (49) become 

r(t) = A(t)X(t) +Z(t)A'(0 + B(0Q(t)B 1 (l) - X(t)C(t)R‘'(t)C(t)Z(t) (55) 


z <>o> = I o • 

After defining the 2n by 2n matrix 

’ A(f) 3(t)Q(l)B'(0 

. C'WR‘V)C(t) - A'(t) 


(56) 


( 57 ) 
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and its 2n by 2n transition matrix 


• W 


'W 
^22 ■ 


( 58 ) 


the closed-form solution of (55), (56) can be written as 

Z W = ( Y N (, 'V J 0 +T !2 (, 'V ) (' 1 22 (, ' , 0 )+ ' i ' 2 l (, ' t O )Z 0 ) ‘' • < 59 > 

In the context of the Hamilton- Jacobi-Bellman theory, where the initial state is denoted 
by X and the initial time by t, the cost (50) then is 

j(G)= J(r,t) = (Y ) ,(t I A)2 + V ]2 (» r t))(>P 22 (t ! ,t)+Y 2I (t r 05:r l . ( 60 ) 

Clearly, J(2,t) of (60) satisfies the boundary condition (8) of Section III on S, 

J(I,t } ) = 1= K(Z, tl ) , (61) 

since Y^Oyt^) = ^22^1'V = 1 and = ^21^1'V = °* * n ° rcler *° verI fy 

Hamilton- Jacobi-Bellman partial differential equation 

|i(I,t) + H(I < i^(I / t),G°(X,^.(2,t),t),t) = 0 , (62) 

the derivatives e £,(R,M ' ) (which is isomorphic to M* ) and 4e>-(X,t) e £(M' ,M‘ ) 

dt n n oi. n n 

are calculated ; 

ii(Z,t)= (-¥ H (t 1 ,t)AI-y ]2 (t ) ,t)C'R' 1 Cr-'i' ]1 (t 1 ,f)BQB'+'F 12 (t 1 ,t)A') 

«(H' 22 (t,,t)+'i' 2 ,(f r t)Xr 1 

- + 4 7 12 (t 1/ t)){'r 22 (t 1 ,t) + ^ 21 (t )/ t)x>' 1 

x(-T 2) (t 1 ,t)BQB'+y 22 (t r t)A'-Y 21 (t r t)AX-'l , 22 (t ) ,t)C‘l!'’cz) 
.hr 22 (r,,0 tv 2 .;t r t;zr' (63) 

■|j(r,t)= 'i' n (t ) ,t)T(i' 22 (t ] ,t) + , r 2) (t I ,t)i)‘' 1 T . 

-(Y n (t ) ,t)I + Y ]2 (t r t))(^ 2 (t 1 ,t)+H’ 21 (t r t)xr V V 

s? 21 (. 1 ,t)T('i’ 22 (t ) ,0+')' 2) (t 1 ,.)X)r , T : (64) 
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where in (63) and (64) (and beiow) the matrices A, B, C, Q, and R are understood to be 
evaluated at time t, and T is the transposition operator (Definition IV.2). 

Combining (53), (54), (63), and (64), it is easily seen that the Hamilton- Jacobi 
Beilman equation (62) is satisfied, since all terms cancel. Hence, by Theorem III. 3 and 
Corollaries 111.5 and 111.6, the Kalman-Bucy filter is a globally superior solution to the 
dynamic infimization problem (48), (49), (50), since the costate P(t) is necessarily a 
positive map. 

Clearly, the above analysis gives the same result for the more general case of 
S(f) f=- 0. Then, the matrix (57) is 

A(t) - B(t)S(t)lf ’(t)C(!) B(l)(Q(t) - S(t)R~ 1 (f)S'(t))B , (0 

.C'(t)R" , WC(t) - A'(f) + C'(t)R‘ , (t)S , (t)B’(r) 

and the analysis goes through in an analogous fashion. 

IV.2. Related Infimization Problems 

The Dual Infimization Problem to the Kalman-Bucy Filtering Problem 
The deterministic linear quadratic regulator problem and the least-squares filtering problem 
for the linear system (4) are often called dual to each other. In the context of non-scalar- 
valued performance criteria, the question arises, in what sense the linear quadratic regula= 
tor problem is dual to the infimization problem of Section IV. 1 with its matrix-valued 
performance criterion (18). 

The control u* : [t ,t^] -»-R m for the n-th order linear system 
x(t) = A(t)x(t) + B(t)u(t) 

x (' 0 } ~ *0 ' 

which is optimal with respect to the cost functional 


(65) 


(a) 

m 
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' 1 

J(u) = x'(t )Fx(t ) + (x'(t)Q(t)x(t) + u‘(t)R(t)u(t))dt 

1 1 

0 

F >0 £ (M',4),- Q(t)> 0 £(M',4), and R(t)£ 0 fi(M' ,«) 

n n m 

(i.e., positive-definite) for all t tf (68) 

is superior in the sense, that it is optimal for every arbitrary initial state Xq £ R n . In other 

words, u* infimizes the function-valued cost J : L^tQ,^) -*(C(R n ),<) ([l9, p.2. Example 3, 

i.e., C(R ) partially ordered by the pointwise total order of (R,<)]), 

r f i 

J(u) = x'^x^FxO^Xq) .'+ (x'^/x^CKOx^/Xq) + u'(t)R(t)u(t))dt (69) 

*0 

The easy verification by the results of Sections II and III is left to the reader 

(see [9, p. 83]). 


The Separation Theorem 

An interesting exercise for the application of the infimum principle is provided by the 
so-called separation theorem for the stochastic linear quadratic Gaussian regulator 
problem [9, Section V.3]. Since the separation of the estimator from the regulator is 
implemented in the constraints of the infimization problem, the analysis of this exercise 
clearly does not prove the separation theorem; it merely verifies the fact, that the 
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IV. 3. An Example with a Set-Valued Cost 

In this section, an optimization problem with a set-valued cost is investigated. 
A superior solution in the sense of the partial order of set inclusion [2, Example 11.4] is 
found. The problem involves a linear uncertain dynamic system, where the uncertainty 
is described by set-membership. 


Statement of the Infimization Problem 


Consider the observed uncertain n-th order linear dynamic system 
x(t) = A(t)x(t) + B(t)(u(t) + w(t)) 


x V =x o 


(72) 

(73) 

y(t) = C(t)x(t) + v(t) . (74) 

B(t) is an n by m matrix of full rank m ^ n, A(t) is n by n, C(t) is r by n, and they all are 
piecewise continuous on [t^t^] . The initial state Xq 6 R , the control uncertainty 
w(.) £ ^(^qA^)/ an d the observation uncertainty v(.) e are merely known to 

belong to a set, i.e., (x Q ,w(.),v(.)) £ S c R n * * The convex constraint 
set S is described by its support functional [l7] / 9-j 6 R / e ^2 ^ 


q 3 (.) * 


r* t - 




(q^(t)Q(t)q 2 (t) + q^(t)R(t)q 3 (t))dtj 


, 1/2 


0 


where F ^0 e (M',4), Q(t)>0 e (M 1 ,4) for all t £ [t n ,t j, and 
n m ... u i 

R(t)£ 0 £ (M^,4) (i.e., positive-definite) for all t £ [t^, t^] . (75) 

In the special case, where F and Q(t) are invertible, this set-membership constraint of the 


uncertainty is equivalent to the "energy constraint" 

nf- 


x 1 F~^x + 

V x 0 v. 


1 


(w'(t)Q(t)w(t) + v‘(t)R(t)v(t))dt < 1 


(76) 


t 


0 
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For t ^[l-Q/t]] find an m by r matrix N*(t) such that the linear output feedback control law 
u(t) = N* (t)y(t) (77) 

generates a set (x(t.j)];^* of uncertain final states x(t^), which is infimal in the sense 
of set Inclusion, i.e., 

J(N*) = {x(t 1 )} N * C J(N) = {x(t 1 )) N for all N(.) . (78) 


Analysis of the Infimization Problem 

With the linear output feedback control lawu(t) = N(t)y(t), (72), (73), and (74) combine to 
x(t) = (A(t) + B(t)N(t)C(t))x(t) + B(t)(w(t) + N(t)v(t)) (79) 

x(t Q ) = x 0 • (80) 

For any particular uncertainty x^, w(.)/ v(.)/ the uncertain final state can be written as 


x(tp = ^(t^t^XQ + 


1 

$(t r t)B(t)w(t)dt +J $(t^,t)B(t)N(t)v(t)dt , (81) 

*0 f 0 

where <$(•/•) is the state transition matrix associated with the systems matrix A(t) + B(t)N(t)C(t), 


or 


X(tj) = L(x 0 ,w,v) , 


(82) 


where L R° x ^ ' s l' near operator defined by (81). With 

the usual inner product on R n x L^t^yt^) x t ^ ) and with the adjoint operator L* of 

L, the support functional <j(q;{x(t^)}| V j), q e R n , can easily be written as [l7l 
. ^(qKx^^ij^) = S> (q;l-{(x 0 ,w,v)}) = ^(L*q;S) 

/ P f l 

= (q 1 (« t r t 0 ) F #^i / f 0 )+ J ^( t ]/ t )B(0Q(0B'(f)<| , (f 1 ,t)dt 


^1 


^(t, ,t)B(t)N(t)R(t)N' (t)B* (t) 4 > * (t, ,t)dt) q 


1/2 


0 

= (q'M^^q) 


1/2 


(83) 



- 42- 


Clearly, £ (M‘ , 4 ) defined in (83) is positive-semidefinite. Furthermore, the 

following partial order inequalities are equivalent : 


<==> 6 (q;{x(t 1 )> N * ) 4 6'(q;{x(t 1 )} N ) 

<=> (t 1 ) lsj . 


(partial order of set-inclusion 
[2, Example 11.4]) 

(partial order of pointwise total order 
of (M) [19, p. 2, Example 3]) 

(partial order of positive-semidefinite 
differences [2, Example 11.3]) 

(84) 


Thus, the infimization problem at hand reduces to the following infimization problem : For 


M(t) = (A(t) + B(t)N(t)C(t))M(t) + M(t)(A(t) + B(t)N(t)C(t))' 

+ B(t)Q(t)B'(t) + B^NWRWN^OB'W (85) 

M(t 0 ) = F , (86) 

find N : [t-,t ] -* M , such that 
U I mr 

M(t^) is infimal with respect to N(.) . (87) 

This infimization problem is quite analogous to the infimization problem (26), 

(27), (28) in the analysis of the Kalman-Bucy filter. Here, the state is M(t) & M‘ and the 

n 

control is N(t) e M . The cost space (M 1 ,4) is the entire state space. As in (29) through 

(41), the costate P(t) e ^((M 1 ,4),(M' , 4 ), 4 ) turns out to be a positive map for all t e [t n ,t ] 

n n U I 

Therefore, the infimization with respect to N(t) of the Hamiltonian 

H = P(t)M(t) (88) 

reduces to the infimization of M(t) with respect to N(t). For the latter, by Theorems 111.1, 
and 111.2, and Remark 111.3 of [ 2 ], the following conditions are necessary and sufficient 


3M 

3N 


= U(M(t)C'(t) + B(t)N(t)R(t))TB(t) = 0 


d 2 M(N,d 2 N) = B(t)dNR(t)dN'B'(t) X0 for all dN/ 0 


(89) 

(90) 


where U and T have been defined in the Definitions IV. 1 and IV.2. Equation (89) is 



( 91 ) 
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satisfied by any matrix N*(t) satisfying 

B(t)N*(t) = - M(t)C'(t)R -1 (t) . 

Since B(t) is of full rank m < n, (90) rs satisfied, and (91) can be written with the 
pseudoinverse B^(t) of B(t) as 

N*(t) = - B t (t)M(t)C'(t)R~ 1 (t) = - (B 1 (t)B(t))” 1 B* (t)M(t)C‘ (t)R -1 (t) . (92) 

As In the case of the Kalman-Bucy filter, the Hamilton-Bellman-Jacobi theory of 
Section lll.l verifies, that N*(t) of (91) and (92) is a globally superior feedback matrix. 
The infimal ellipsoidal set {x(t^)}^ it of all possible final states is described by the support 
functional 

^(q;{x(t ] )} N *) = (q , M*(t 1 )q) 1 ^ 2 , (93) 

where M*(t^) can be calculated from the matrix Riccati differential equation 
M*(t) = A(t)M*(t) + M*(t)A'(t) + B(t)Q(t)B'(t) 

- M*(t)C(t)R“ T (t)C(t)M*(t) (94) 

M*(t Q ) = F . (95) 



V. CONCLUSIONS 


This research has been concerned with superior solutions to optimal control problems 
with non-scalar-valued performance criteria. The main contribution to the theory of optima 
control is the infimum principle in Section II, which constitutes necessary conditions for a 
control to be superior with respect to a non-scalar-valued performance criterion attaining 
its value in a finite-dimensional abstract partially ordered cost space, the positive cone of 

i • 

which is closed and has nonempty interior. Further contributions include the sufficiency 
results for a control to be superior in Section III, most notably the extension of the 
Hamiltjon-Jacobi-Bellman theory to the case of non-scalar-valued performance criteria. 

In Section IV, the applicability of the theory of Sections II and III has been demonstrated 
with two non-trivial examples, viz. the Kalman-Bucy filter and an uncertain optimal control 
problem with set-membership description of the uncertainty. 

In order to keep the proofs simple, the cost spaces have been restricted to be 
finite-dimensional. Conceptually, the theory of this paper can be extended to infinite- 

I •• 

dimensional cost spaces, provided their positive cones are endowed with suitable topological 
properties. The additional properties needed have been discussed in [16], where a theory of 
superior solutions to problems of mathematical programming in abstract infinite-dimensional 
partially ordered spaces has been published. 

In the theory of optimal control for non-scalar-valued performance criteria, the 
problem of existence of superior solutions still is open to further research. Presently, it 
is unclear what typesof reasonable assumptions are needed in the problem statement, in 
order to obtain existence of superior solutions rather than existence of noniferior solutions. 

The major difficulty in the existence problem has been described in [2, Example IV.5 and 

) . . • ■ 

Section V, last paragraph]. 
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APPENDIX 


Definition A.l. A map f from the linear poset (X,*4) into the linear poset (Y,4) is called 


order-positive/ if x ^ 0 e (X,4) implies f(x) > 0 6 (Y,4). Furthermore, g£((X,4),(Y,4),4) 
denotes the linear poset of all linear maps from X into Y, which is partially ordered by 
the coneofall positive linear maps. 


Lemma A. 2. The positive cone of the directed linear poset (X,4) is the supremal [2, 
Definition 11.5] set (in the sense of the partial order of set-inclusion) of the subsets of 
(X,4), which are mapped into the positive cone of the linear poset (Y,4) by all positive 
linear maps. 
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